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<Zn.S  > are  equi  valent  if  and  only  if  there  is  a certain  homomorphism  between 
the  matrix  rings  generated  by  Q(t),  t e fO,“J  and  Y(t),  t e (0,“}.  The 
equivalence  is  identical  to  weak  lumpability  in  the  case  where  , Sn ^ is  a 
renewal  process.  1 — <--5^ l 4 H 


Although  the  conditions  for  strong  lumpability  can  be  written  in  an 
attractive  form,  they  are  too  restrictive  to  be  of  any  real  interest.  Weak 
lumpability  is  of  more  interest  since  (as  will  be  shown)  it  occurs  in  less 
trivial  examples,  but  the  necessary  conditions  are  very  complicated.  The 
equivalence  defined  herein  has  the  advantage  of  having  simple  necessary  and 
sufficient  conditions. 
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Abstract 


We  define  a form  of  equivalence  between  Markov- renewal  processes  that 
includes  strong  and  weak  lumpability  as  special  cases,  and  examine  its  properties. 

If  {X  ,T  } is  a Markov-renewal  process  with  kernel  Q(t)  and  {Z  ,S  } is  a 
n n n n 

Markov-renewal  process  with  kernel  Y(t),  then  it  is  shown  that  {Xn,Tn}  and 
{ Zn, Sn } are  equivalent  if  and  only  if  there  is  a certain  homomorphism  between 
the  matrix  rings  generated  by  Q(t),  t e [0,°°]  and  Y(t),  t e [0,"].  The 
equivalence  is  identical  to  weak  lumpability  in  the  case  where  {Z  ,Sn)  is  a 
renewal  process. 

Although  the  conditions  for  strong  lumpability  can  be  written  in  an 
attractive  form,  they  are  too  restrictive  to  be  of  any  real  interest.  Weak 
lumpability  is  of  more  interest  since  (as  will  be  shown)  it  occurs  in  less 
trivial  examples,  but  the  necessary  conditions  are  very  complicated.  The 
equivalence  defined  herein  has  the  advantage  of  having  simple  necessary  and 
sufficient  conditions. 


CHAPTER  1 


1.  Introduction.  A random  process  (xn>Tn)  n= 1,2,3,* ••  with  taking  values 
in  a finite  or  countable  set  S (called  the  state  space),  and  T taking  values 
in  [0,°°]  is  called  a Markov  renewal  process  (MRP)  if 

P(Xn+l  = j’  Tn+1  - ^ X0’X1’  ”'»Xn*Tl*T2’ '"*Tn)  = P(Xn+l  = j’  Vt-l-^V 

for  all  n e Z+,  j e S,  t e [O,00].  Markov  renewal  processes  arise 
naturally  in  queueing  systems  and  since  renewal  processes  and  Markov  chains 
are  special  cases  of  MRP's,  a large  class  of  problems  in  the  study  of  random 
processes  can  be  handled  with  Markov  renewal  theory. 

Consider  the  departures  from  an  M/G/l/N  queue.  Let  Tr  be  the  time  between 
the  (n-l)St  and  n^1  departure,  and  let  be  the  number  of  customers  in  line 
the  instant  after  the  n*"*1  departure.  It  is  well  known  [3  ] that  (X  ,T  ) is  a 
MRP  on  a state  space  consisting  of  {0, 1,2, • • • ,N} . Now  consider  the 
special  case  where  G = M.  Since  the  M/M/1  queue  is  an  M/G/l/N  queue,  the 
departure  process  is  a MRP  with  a countable  state  space.  By  (1]  and  [3], 
though,  we  know  that  in  steady  state  the  departure  process  from  an  M/M/1 
queue  is  a Poisson  process,  which  like  any  renewal  process,  is  a one  state  MRP. 
Thus,  in  some  sense,  the  infinite  state  MRP  that  represents  the  output  from  an 
M/M/1  queue  is  equivalent  to  a Poisson  process.  An  enormous  amount  of  work  lias 
been  done  on  systems  with  M/M/1  queues  that  never  would  have  been  possible 
were  it  not  known  that  the  output  from  an  M/M/1  queue  is  a Poisson  process.  Any 
time  it  can  be  shown  that  a MRP  is  "equivalent"  to  a renewal  process,  the  amount 
of  computation  necessary  to  make  statements  about  the  process  will  be  drastically 
reduced.  This  paper  is  a first  step  towards  getting  such  results. 
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When  two  random  processes  are  called  equivalent  in  this  paper,  it 
means  that  certain  specific  conditions  (to  be  given  later)  are  satisfied 
by  the  two  processes.  The  conditions  are  strong  enough  to  be  of  interest, 
and  weak  enough  to  assure  that  there  are  plenty  of  examples. 


2.  Lumpabilitv.  Probably  the  simplest  case  of  equivalence  between  MRP's  is 

lumpability  in  Markov  chains  [8],  Let  {Xq}  be  a Markov  chain  on  a finite  or 

countable  state  space  S.  Let  A^,A2»  • • *,  A^  be  a partition  of  S,  and  let 

F : S -*•  {A^,A2,  "’.A  } be  the  map  that  "lumps"  the  state  space  S onto  the 

partition  {A^,A,,  ••■.A^}.  The  process  {F(Xn>}  may  or  may  not  be  a Markov 

chain.  In  general,  the  probability  of  going  from  Ai  to  A^  in  {F(Xn>}  will 

depend  on  precisely  which  element  of  A.  the  (X  } process  is  in.  If  for  each 

l n 

i and  j,  though,  the  probability  of  going  from  A^  to  A^  is  independent  of  the 

State  in  A^^  that  the  {Xn>  process  is  in,  then  the  process  {F(Xn)}  is  a Markov 

chain.  When  this  happens  we  say  {X  } is  strongly  lumpable  to  {F(X  )}.  This  is 

n - — — — ■ ■ n 

a special  case  of  the  equivalence  to  be  defined. 

For  example,  say  S * {1,2,3}  and  let  {Xn}  have  transition  probability 
matrix 


Let  F(l)  ■ A^,  F(2)  - F(3)  ■ A2.  The  process  { F (Xn) } is  a Markov  chain  on  (A^ ,A,} 


with  transition  probability  matrix 


If  {X^}  is  strongly  lumpable  to  {FCX^)}  then  no  matter  which  state  in  S 

the  process  starts  in,  {F(X  )}  will  be  a Markov  chain.  In  fact,  even  if  the 

n 

precise  state  that  the  process  begins  in  is  not  known,  the  ensuing  (F(Xn)} 
process  is  a Markov  chain. 


Sometimes,  even  though  {Xn}  is  not  strongly  lumpable  to  {F(Xn)},  the  process 
{F(X^)}  is  a Markov  chain  when  {Xn}  is  in  steady  state.  When  this  happens  we 
say  {Xn}  is  weakly  lumpable  to  {F(Xn)}. 

If  S is  a finite  set  with  m elements  and  F(S)  has  n elements  A^,A2»  •••,A 
(n  < m) , then  the  following  mxn  matrix,  U,  can  be  constructed.  Let 


( 0,  if  i 4 Aj  , 

\ l,if  i c A.  . 


If  {X  } has  a steady  state  then  there  is  a vector  II  that  satisfies  HP  = II 
n 

where  P is  the  transition  probability  matrix  for  {Xn>.  Let  H be  an  n * m 


matrix  with 


0,  if  j 4 At  , 


, if  j e A, 


The  iC  row  of  n is  the  conditional  probability  of  being  in  state  j given  that 


the  process  is  in  steady  state  and  that  the  process  is  in  A^., 
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Kemeny  and  Snell  [8  ],  show  that  {Xn)is  strongly  lumpable  to  (F(Xn)}  if 

and  only  if  PU  = U(T!py),  and  that  if  {F(X  )}  is  a Markov  chain  then  its 
~ n 

transition  probability  matrix  is  HPU.  They  also  show  that  IIP  = (IIPU)5  or 
PU  = y(nPU)  is  a sufficient  condition  for  (X^}  to  be  weakly  lumpable  to  { F (X^) } , 
For  example  let  S = (1,2,3}  and  set  F(l)  = F(2)  = F(3)  = Suppose  the 

transition  probability  matrix  for  {X^}  is 


In  this  case  II  = ( -j  , , -J  ) so 


1/2  1/2/  and  U = 


(X  } is  not  strongly  lumpable  to  (F(X  )}  since  P(F(X  ) =A1|F(X  ,)  =A9) 

n n n j-  n— x 

depends  on  whether  X^  ^ is  equal  to  2 or  3.  This  can  be  seen  formally  by  noting 
that  PU  ^ U(nPU).  In  steady  state,  though,  (F(Xn)}  is  a Markov  chain  since 
IIP  = (HPU)n  . The  resulting  Markov  chain  {F(Xn>}  has  a transition  probability 


matrix 


npu  = 


1/2  1/2 

1/4  3/4 


The  necessary  conditions  for  weak  lumpability  are  much  less  appealing  than 
the  necessary  and  sufficient  condition  for  strong  lumpability  or  the  sufficient 
conditions  for  weak  lumpability.  If  y is  a probability  vector  on  S then  define 
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[ Y ] to  be  the  vector  of  conditional  probabilities  of  being  in  state  j 


(j  *1,2,  • • •,  m) , given  that  the  process  is  in  For  example,  the  1 row 

of  the  matrix  IT  is  [II]1.  Let  I\  be  the  set  of  all  finite  sequences  of  states 

in  F (S)  that  end  with  A..  If  A.  , A.  A.  , A.  and  A.  , A.  , •••  A.  , A. 

3 H x2  V 3 31  32  Jq  J 

are  two  elements  of  I\  then  for  (X^)  to  be  weakly  lumpable  to  {FCX^)}  it  must 
be  true  that  for  each  a t(l,2, 


L J.  pib  n1  - I.  J.  pi6  ■'i 


i € S S € A 


itS  S e A 


where 


1 ^1  x2  ^3  ^4  3 

y - I •••  [ [ [ [ [n]  p]  p]  p]  4 •••  p]  k p] 


>>  i?  ii  3*  j j 

y = [ •••  [ [ [ [ [n]  P]  “ P]  J P]  4 •••  P]  q P] 


Serfozo  [10]  showed  that  strong  and  weak  lumpability  can  be  defined  for 
MRP's  in  an  analogous  manner.  In  fact,  the  conditions  for  strong  and  weak 
lumpability  in  MRP's  are  virtually  identical  to  the  conditions  for  Markov  chains. 

If  (X^T^)  is  a MRP  on  a finite  state  space  S = (1,2,3,  •••m},  with 
kernel  Q(t)  (i.e.  ( t)  = P(Xr+1  = j,  Tn+1  <_  t|Xn  = i))  and  F : S -*•  (A^Aj,*  • • , An} 

is  a partition  of  the  state  space  then  {Xn,Tn}  is  said  to  be  strongly  lumpable 
to  (F(Xn),Tn>if  {F(Xn),Tn}  is  a MRP. 

Again,  let  n be  the  steady  state  vector  for  the  embedded  Markov  chain 
(i.e.  iTQ(»)  = P) , and  let  p,U  be  defined  as  before.  Serfozo  shows  that 
{Xn,Tn}  is  strongly  lumpable  to  (F(Xn) ,Tn>  if  and  only  if  Q(t)U  = U(pQ(t)y) 
for  all  t e [0,«],  Likewise  if  for  all  t,  Q(t)U  ■ U(PQ(t)U)  or  PQ(t)  = (PQ(t)  y)P 
then  {F(Xn),Tn}  is  a MRP  in  steady  state  (i.e.  weakly  lumpable).  Unfortunately, 
the  necessary  conditions  for  weak  lumpability  are  again  very  complicated. 
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Let  I\  be  the  set  of  all  finite  sequences  of  states  in  F(S)  that  end  with  A^  . 
If  A^,  » *'-,Ai  , Aj  anc*  Aj q > Aj ^ > • • • > Aj  > Aj  are  two  elements  of  I\  and 

{t^,  t9,  • • • , t^} , {s^,  s2,  • • • , s } are  two  sequences  of  positive  real  numbers 
then  for  txn»Tn^  t0  be  weakly  lumpable  to  {F(Xn),T^}  it  must  be  true  that  for 
each  a e (1.2,  •••,n)  and  t e [0,°°], 


I l Qie(t)  y\  = I I ^i8(t)Yi 


i e S S e A 


i e S 6 e A 


where 


Y1  = [ •••  [[[[[n]  1 Q(tl)]  2 Q(t2) ] 3 Q(t3) ] 4 * * * Q(tk_x) ] k Q(tk)]J 


Y2  = [ •••  [ t [ [ [ n] J 1 Q(S;l)]J2  Q(s2)]J3  Q(s3)]J4  - * * Q(sq_1)  ] Jq  Q(s  )]\ 


In  this  paper  a type  of  equivalence  will  be  defined  that  includes  all  of 
the  cases  discussed  so  far  and  has  the  added  property  that  a necessary  and 
sufficient  condition  for  two  MRP's  to  be  equivalent  can  be  written  in  a simple 
form. 

3.  Definitions  and  Preliminaries.  The  following  concepts  will  be  used 
throughout  this  paper. 

Definition  1.1.  Let  V be  a vector  space,  and  let  T : \J  -*■  \J  be  a function  on 
V . A subspace  W of  1/  is  said  to  be  invariant  with  respect  to  T if  for  all 
w e W,  Tw  e W. 

For  example,  say  l / is  Rn  and  T(v)  = Av  where  A is  an  n * n matrix.  If 
wi,w2,  • • •,  w^  are  eigenvectors  of  A,  then  the  space  U1  spanned  by  w3,w2>  •••,w^ 
is  invariant  under  T. 


Definition  1.2.  A ring,  R is  a collection  of  objects  along  with  two 


operations  +,  • that  satisfy  the  following  properties:  ya.b.c  e R, 

(1)  a + b e R 

(2)  a + b = b + a 

(3)  (a  + b)  + c = a + (b  + c) 

(4)  3 0 e R that  satisfies  a + 0 = a 

(5)  3 -a  that  satisfies  a + (-a)  = 0 

(6)  a • b e R 

(7)  a • (b-c)  = (a-b)  • c 

(8)  a * (b  + c)  = (a*b)  + (a*c) 

(9)  (b  + c)  • a = (b-a)  + (c-a) 

The  operation  • need  not  be  commutative  in  a ring.  For  example,  the  set 
of  all  n x n matrices  is  a ring.  In  this  paper  we  will  be  interested  in  various 
subsets  of  the  set  of  all  n x n matrices  that  retain  all  the  ring  properties. 

Definition  1.3.  Let  (a^},  i e I,  be  a collection  of  elements  of  a ring,  R. 
The  ring.  A,  generated  by  (a^ } is  the  smallest  subring  of  R that  contains  all 
of  the  { ai  }. 

For  example,  consider  the  ring  of  integers  Z.  The  ring  generated  by  {2} 
is  the  ring  of  even  integers.  For  a less  trivial  example  consider  the  ring, 

N,  of  nxn  matrices.  Let  M^,M2,  • • • , be  elements  of  N.  A typical  element 

2 5 3 

of  the  ring  generated  by  • • • , might  be  Mpl^  + M6M2  - M^. 

Definition  1.4.  Let  R^  and  R2  be  two  rings  and  let  T : R^  - R,  be  a map. 


T is  called  a ring  homomorphism  (homomorphism)  if,va,b  e R^ , 


Q 


(1)  T(a-b)  = T (a)  • T(b)  and 

C)  T (a  + b)  * T (a)  + T(b). 

Consider  the  following  example  of  a ring  homomorphism.  Let  y be  some 
vector  in  Rn  and  let  be  the  set  of  all  n * n matrices  that  have  y as  a 

left  eigenvector.  If  A e R^,  define  T(A)  = a where  a is  the  eigenvalue  of  A 
associated  with  y.  Thus  T(A  + B)  = T (A)  + T (B)  and  T (AB)  = T(A)T(B)  so  T is 
a homomorphism  from  R^  to  R. 

In  this  paper  we  will  only  consider  a special  class  of  MRP's  defined  as 
follows. 

Definition  1.5.  An  m state  MRP,  {Xn,T  },  1 <_  m <_  »,  with  kernel  Q(t) 

will  be  called  simple  if 

(1)  Q^t)  is  nonnegative,  nondecreasing  and  right  continuous, Vi, j , 


(1) 

Qi^(t)  is 

m 

(2) 

I Q,-  4 (* 

1J 

J*1 

(3) 

<0 

t_i. 

rr 

II 

(4)  3!'  n e R|J_  that  satisfies 


(4a)  HQ(«)  = n 

(4b)  HU  = 1 


(4c)  lim  0 (“)  = U n 

n -*•  00 


where  U = (1,1,  • • •,  1) 1 . 


P 


Conditions  (1),  (2),  and  (3)  assure  that  Q(t)U  is  a column  of  nonnegative 
distribution  functions.  Condition  (4)  is  equivalent  to  requiring  that  the 
embedded  Markov  chain  {X^}  is  irreducible,  aperiodic  and  recurrent  non-null. 

(See  [2]  for  proof  of  this  assertion  and  other  similar  results.) 

Let  {X^.T^,}  be  an  n state  MRP  with  kernel  Q(t)  and  say  the  initial 
distribution  on  the  state  space  is  II.  The  following  quantities  are  of  interest. 

(1)  P(T1  < tjXQ  * i) 

(2)  P^  £ t) 

(3)  P(TX  < tr  T2  < t2,  •••,Tn<  tffi) 

We  can  write  (1)  as 

n 

(4)  < t|XQ  = i)  = l P(XX»  j,  T1  < t|x0  = i)  * (Q(t)U)i  . 

j = l 
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In  Chapter  II  we  define  equivalence  between  MRP's  and  investigate  its 
properties.  Sections  1-4  deal  with  the  important  special  case  where  the 
equivalence  is  between  a MR?  and  a renewal  process  (a  one  state  MRP) . 
Section  5 deals  with  equivalence  between  finite  state  MRP's. 

CHAPTER  II 


1.  Equivalence.  A recurrent  renewal  process  {Sn)  is  a sequence  of  independent 
and  identically  distributed  nonnegative  random  variables  with  sn  < 00  with 
probability  one.  The  sequence  {S^}  can  be  thought  of  as  the  times  between  soma 
fixed  event  that  occurs  repeatedly.  Associated  with  each  MRP,  {X^T^},  is  a 
sequence  (Tn).  Suppose  {Xn,Tnl  is  a simple  MRP  on  a finite  state  space 
S = {1,2, • * * ,N}  with  kernel  Q(t).  If  Y is  the  initial  distribution  on  S then 

(1)  P(T^  <_  t)  = yQ(t)U  where  U = (1,1,  •••,1)T,  and 

(2)  P(T1  £ tr  T2  £ t2,  •••,Tn  1 tn)  = YQ(t1)  Q(t2)  •••  Q(tn)U. 

Let  II  be  the  steady  state  vector  associated  with  the  embedded  Markov  chain 

{X  },  and  define  r(t)  = IIQ(t)U. 
n 

Leima  2.1.  r(t)  is  the  cumulative  distribution  of  some  nonnegative 
random  variable. 

Proof . From  conditions  (1)  and  (3)  of  Definition  1.5,  r(t)  = 0 if  t<  0 
and  r(t)  is  nondecreasing  and  right  continuous.  From  condition  (2)  we  have 
r(“)  = nQ(°°)U  * Hu  ■ 1 so  r(t)  is  a distribution  function  of  a nonnegative 


random  variable.  q 
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For  the  remainder  of  this  chapter  all  MRP's  will  be  assumed  to  be  simple, 
and  all  the  renewal  processes  will  be  recurrent.  The  following  theorem 
motivates  the  definition  of  equivalence  between  a MRP  and  a renewal  process. 

Theorem  2.2.  If  {X  ,T  } is  in  steady  state  then  {T  } is  a renewal  process 
n’  n n r 

if  and  only  if  Vm,  Vt^t.,,  • • •,  tffi,  IIQ(t1)Q(t2)  "Q(tm)U  » r(t1)r(t2)  ••r(tffi). 

Proof.  ( if  (T  ) is  a renewal  process  then  P(T.  < t,  , T-  < t,,**\T 
— - — n x — x z — c 7 rn 

» P(T.  < t.)  P(T,  < t,,)*’*  P (T.  < t ).  But  this  savs 
1 — i.  1 — x x — m 

HQ(T1)  Q(t2)  • • •Q(tm)U  = r(t1)  r(t2)  • • *r(tm) . 

( ~ ) We  must  show  that  {T^}  is  a sequence  of  nonnegative  independent  and 

identically  distributed  random  variables.  Since  Q(t)  > 0 it  is  clear  that  {T  } 
is  nonnegative.  Also  r(t)  is  not  a function  of  n,  so  it  suffices  to  show  that 
{Tn}  is  a sequence  of  independent  random  variables.  Let  i^,i2,  •■•in  be  ar.v 
n positive  integers.  We  must  show  that 


P(T.  < t.  , 
*1  X1 


:i,iCi 


•,Ti  < t.  ) * P (T 


il  11 


P<Tl2X'l2> 


P(T  <t,  ) 
xn  xn 


Since 


p<Ti1iti1- 


<.t. 

) l* 


Ti  ±Ci  > 
n n 


P^  < ti« 


^2  — c2  ’ 


.T, 


1 V 


,Tin  — 


where 


t4  , if  j = 
xk 


-»  if  1 * ^1  * i2 * ***  in)  * 


we  have  ^l-H’  T2  - c2» " ’ ’»  Ti  -ci  > * r(tL)  r(t2)  ••• 

r(tx)  r (tp)  r(tin)  * r(ttl)  r(tio)  ••  r(tin)  since  r(»)  - 1. 
a renewal  process.  □ 


r(t,  )•  But 
1n 

Thus  {Tn}  is 
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From  now  on  the  following  notation  will  be  used.  The  symbol  0 will  denote 
a simple  MRP  with  kernel  Q(t)  . The  set  of  matrices  { Q ( t ) } , t c [0,=°],  along  with 
an  initial  probability  distribution  on  the  state  space,  describe  the  MRP  in 
question,  since  from  them  it  is  possible  to  determine  all  transition  probabilities. 
Thus,  there  is  no  ambiguity  in  using  the  symbol  Q to  denote  a MRP.  Likewise,  the 
symbol  r denotes  the  renewal  process  with  distribution  r(t)  without  ambiguity. 

Definition  2.1.  Let  Q be  a simple  MRP  with  steady  state  vector  ",  and  let 
r be  a renewal  process.  Then  Q is  equivalent  to  r © -r)  if 


(3)  Vn,  tr  t2  •••  tn,  -QUj)  Q(t2)  •••  Q(tn)U  - r(tj_)  r(t,)  •••  r(tn) 


By  theorem  2.2,  Q is  equivalent  to  a renewal  process  if  and  onlv  if  {T  } is 

n 

a renewal  process  in  steady  state.  Thus  equivalence  and  weak  lumpability  are 
the  same  thing  when  the  lumping  is  to  one  state.  From  the  definition  it  is 
clear  thatQ-r  implies  that  r(t)  * HQ(t)U,  which  is  the  steady  state  distribution 
of  the  time  between  state  transitions  in  Q.  The  reason  that  the  steady  state 
vector  T.  is  used  in  the  definition  comes  from  the  following  theorem. 

Theorem  2.3.  Let  Q be  a simple  n state  MRP  (m  _<  °°) . If  there  exists  a 
probability  vector,  Y,  and  a renewal  process,  f,  such  that 

Vn,  t1,t2,---,tn,  YQ(t1)Q(t2)---Q(tn)U  - f(t1)f(t2)---f(tn), 

then  f(t)  - r (t) , Vt  (i.e.  YQ(t)U  - HQ(t)U,  Vt) . 


Proof . We  split  the  proof  up  into  two  parts. 


(case  1:  m < “) . We  know  that 

f=] 

lim  Qn(®)  - UTI  | , 


and  so  Vc  > 0,  3N  such  that  if  n > S, 
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nax  Q“  (x)  - " < e/m.  Thus  the  jCh  column  of  Qn  1(«)  is  (a,, a,,..., a )* 

. 4 lj  J 1 2 tn 

» j 

where  a^  - 1 < c/m  for  each  i.  Thus 


^L'^'  "i  - e/"  , V'k  • c/m- 


This  says  that  each  element  of  the  vector  yQ  («)  differs  from  the 
corresponding  element  of  H by  less  than  c/m.  By  hypothysis, 

>Q(t1)Q(t2) • *Q(tn)U  - f(t1)f(t7)-*f(tn),  so  by  letting  t^  * t2  * • * = t^_1  =*  * 

we  getyQn~^(®)  Q(tn)U  “ f (tn) . Let  yQn  *(®)  « H'.  To  show  that  f(t)  * r(t)  it 

suffices  to  show  that  [ ( t )U  - HQ( t )u|  is  small. 

n n 

r/Q(tn)U  - EQ(tn)U|  * in  - riQ(to)U  < c/m  UTQ(tn)U  < e/m  UTC  - t. 

Since  e was  arbitrary  we  have  r(t)  * f(t). 

00  <30 

(case  2:  m = “) . Fix  e and  choose  N such  that  £ y.  < e/7,  £ H.  < e/7. 

i=*N+l  i*N+l 

Since  Q is  simple  we  have  lim  Q?..  (“)  ■ II  Vi,  so  we  can  choose  M such  that  n > M 

n-*-»  ■LJ  J 

implies  |Qij(°°)  - H ^ j < e/7N  for  i,j  £ N. 

We  must  show  |f(t)  - r(t)|  is  small, Vt.  Since  f(t)  » yQn(®)Q(t)U  for 
any  value  of  n,  it  suffices  to  show  that  for  n > M,  | yQn(0C)Q(t)U  - r(t)  | is 
small, Vt.  Since  r(t)  - HQ(t)U, 


|yQn(«)Q(t)u  - r (t)  | < | yQn(»)  - n|Q(t)U  < l UqV)  - H| 


L I 

J-i 

We  have, 


I yQU (°°)  - E!  + 1 |yQn 


Q (-)  - n .. 


j»N+l 


N 


I ! YQn(°°)  - n|.  <.  I l yJq”  (»)  - n.|  - l y±  l |q“  <->  - n.j 

j*l  J j*l  1*1  X J 1 jal  J 


l 


N N N 

£ 1 + e/7  - I l y.qJ. (»)  1 1 + e/7  - [ l y.q"  («) 
j-1  i-1  1J  j-1  i-1  1 1J 


N N N N 

£ 1 + e/7  - l l Y, (H.  - e/7I0  = 1 + e/7  - T Y T (r  - e/7N) 

J-1  1-1  J i-1  J-1  j 


£ 1 + e/7  - (l-e/7) (1  - e/7  - e/7)  - 1 + e/7  - 1 + 3e/7  - 2e2/49  < 4e/7. 


Thus  |f  (t)  - r (t)  j <_  l | vQn(»)  - n|,  <.  3e/7  + 4e/7  = e.  o 

J-1  J 


2.  Conditions  for  Equivalence.  We  are  now  ready  to  find  the  conditions  for 


Theorem  2.4.  Let  Q be  a MRP  with  steady  state  vector  II,  and  let  r be  a 
renewal  process.  If  Vt,  HQ( t)  ® r(t)JI  then  Q~ r. 


Proof.  If  HQ(t)  = r(t)H  then  Vn,  t, , t„,  • • •,  t , 

l i n 

HQ(t1)  Q(t2) — Q(tn)U  = r(t1)  HQ(t2)  •••  Q(tn)U 


-r(t1)  r(t2)  •••  r(tn)nu  = r(tx)  r(t0)  •••  r(tn>.  Q 


1 
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Theorem  2.5.  If  Vt,  Q(t)l'  * U r(t)  then  Q~r. 

Proof.  If  Q(t)U  - U r(t)  then  Vn , t.,  t„  t , 

~Q ( tx)  Q(t,)  •••  Q(tn)U  - HQ ( t : ) Q ( 1 7 ) •••  Q(tn_1)U  r(tn) 

■ ITUrCt^  r (t9)  • • • r(tn)  » rft^  r(t2>  r(tn>.  0 

Theorems  2.4  and  2.5  are  special  cases  of  the  sufficient  conditions  for 
weak  lumpability  that  Serfozo  gives  in  [10].  Theorem  2.4  says  that  if 
the  steady  state  vector,  n,  is  a left  eigenvector  of  Q(t)  for  every  t then  Q~r, 
where  r(t)  is  the  eigenvalue  of  Q(t)  corresponding  to  the  eigenvector  H.  Notice 
that  in  theorem  2.5  it  was  not  important  that  the  starting  vector  was  H.  Any 
vector  that  satisfied  YU  “ 1 would  have  worked.  This  is  because  Q(t)U  = Ur(t) 
is  a necessary  and  sufficient  condition  for  strong  lumpability  of  Q to  r. 

Theorem  2.5  says  that  if  the  row  sums  of  the  matrix  Q(t)  are  the  same  for  all 
t then  Q-r  where  r(t)  is  the  common  value  of  the  row  sums  of  Q(t).  If  the  row 
sums  of  Q(t)  are  the  same,  then  no  matter  which  state  the  process  is  in,  the 
time  until  the  next  transition  has  the  same  distribution.  Thus,  knowing  the 
state  that  the  process  is  in  gives  no  extra  information  about  the  time  until 
the  next  transition.  It  is  clear  that  in  such  cases  the  times  between  state 
transitions  is  a renewal  process.  The  intuitive  justification  for  theorem  2.4 
is  less  obvious,  but  most  interesting  cases  of  equivalence  seem  to  be  of  that 
type.  We  will  see  later  that  Burke's  theorem  is  a simple  corollary  of  theorem.  2.4. 

Let  Q be  a MRP  with  n states  (1  < n <_  ®) , and  steady  state  vector  ".  We 
define  the  following  subsets  of  Rn. 
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Definitions  2.2. 

(A)  Let  5 = {v  e Rn:  vQ(t)U  = (vU)  r(t),  Vt}  where  r(t)  = 7Q(t)U. 

(B)  Let  1/  be  the  largest  subset  of  S that  is  invariant  under  multiplication 

by  Q(t)  (i.e.  v e l/=^*Vt,  vQ(t)  e 1/ ) . 

(C)  Let  P - (v  e Rn:  v > 0,  vU  ■ 1}.  P is  the  set  of  probability  vectors. 

(D)  Let  K * 1/rtP  . 

Lemma  2.6.  S and  V are  subspaces  of  Rn.  K is  a compact  and  convex  set 
if  n < ®. 

Proof . Clearly  {H}  e S and  {0}  e S,  so  S has  at  least  two  elements.  Say 
Y^»  Y 2 e Then  Y1Q ( t)U  * r(t)(YjU)  and  Y2Q(t)U  = r(t)(y2U).  Thus 
(a  Y^  + by0)Q(t)U  = r (t)  (a  y^  + b y2)U  so  a y^  + b y2  e 5.  Let  I'J  be  any  invariant 


subset  of  S and  let  W*  be  the  subspace  generated  by  W.  Say  w e W*. 


I a.,  v.  where 
i=l 


k „ 

. w . wuci  c w.  e W i * 1,2,  •••  k.  But  wQ(t)  = J a.w.Q(t)=  J a.  w! 
^ i ^ i=l  ^ ^ i ^ 


k 

I 

i=l 


where  w^  e W since  k1  is  invariant.  Thus  wQ(t)  e W*,  so  W*  is  invariant.  Since 

V is  defined  to  be  the  largest  invariant  set,  it  must  be  a subspace. 

The  set  P of  probability  vectors  in  Rn  is  closed  convex  and  bounded.  So  is 

PDI/  since  1/  is  a subspace,  thus  K is  compact  and  convex.  O 


Consider  the  column  vector  Q(t)U.  If  v e S then 


(vU) 


Q(t)0  = HQ(t)U,  Vt. 


This  says  that  ( ~ Q(t)U  =*0,  Vt,  which  says  that  the  vector  — ~ 

is  orthogonal  to  the  vector  Q(t)U,  Vt.  Thus  we  have  the  following  important 
lemma . 

Lemma  2.7.  If  Q is  an  n state  MRP  and  there  exists  times  t,,t„,  •••  t 
1 2’  n 

such  that  {Q(t^)U,  0(t7)U,  '•*  Q(tn)u)  is  a linearly  independent  set  then  S 
is  the  subspace  of  Rn  generated  by  {H}  . 


F 
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Proof.  If  v t S then  ( - ~)  must  be  orthogonal  to  each  Q(t)U.  But 

only  the  zero  vector  can  be  orthogonal  to  n independent  vectors  in  R . Thus 
the  only  elements  of  S are  the  points  on  the  line  through  H and  the  origin.  O 

We  now  show  the  importance  of  the  sets  F and  K . 

Theorem  2.8.,  Q~  r <-'■  >>  li  e l/. 

Proof.  (=^  ) If  Q-r  then  Vn  t,  ,t,,"  • ,t  , nQ(t.)  O(t-)  • • • 0(t  )U 
, i z n l z n 

= r(t^)  r(t2)  •••  r(tn).  Say  IIQft^)  Q(t9)  • • • Q(tn)  i S . Then  3t  such  that 

7.Q(t1)  Q(t2)  •••  Q(tn>  Q(t)U  i (HQ(t^)  •••  Q(tn)U)r(t).  But  this  says  that 

Q*r.  Thus  Vn,  t. , t,  ,• ••  ,t  we  have  -TIQ(t1  ) Q(t»)  • • • Q( t ) t S.  Let 
i z n l z n 

W = {w:  w = KQ(t..)  Q(t0)  • • • Q(t  ) for  some  n,t.  ,t~,  **•  ,t  }.  W must  be  a 
i - n x ^ n. 

subset  of  S,  and  clearly  W is  invariant  under  multiplication  by  Q(t).  Thus 
I'J  tZ.  \J  . But  II  e W since  nQ(<“)  = II,  so  II  e l/. 

( ..  ) If  Ti  e V then  .T  e K.  Let  v e K.  Since  each  element  of  Q ( t ) is 

nonnegative,  vQ(t)  _>  0*  Also  vQ(t)  = v'  where  v'  z V . Finally,  vQ(t)U  =r(t) 

since  v e K so  v'U  = r(t).  This  implies  that  v(^yyy  ) e K,  so  K is  invariant 

under  multiplication  by  Q^cVr(t).  Thus  IlQ(t^)  0(t0)  •••  QCt^JU  = r (t^)  r(t2)  • • ■ r (t^ 

Corollary  2.8.1.  Q-r  > II  c K. 

Proof.  If  II  e K then  K e \J  so  0-r.  If  Q-r  then  H e F . 3ut  HU  = 1 so 
n e K.  a 

Theorems  2.4  and  2.5  were  sufficient  conditions  for  Q-r  which  are 
relatively  easy  to  use  in  practice.  Theorem  2.8  gives  a necessary  and 
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i 

sufficient  condition  but  there  is  not  yet  any  simple  way  of  determining  what 
t'  is.  The  following  theorem  is  a necessary  condition  for  Q~r  which  is  also 
useful  in  practice. 

Theorem  2.9.  If  Q is  a finite  state  MRP  then  0~r  =^r(t)  is  an 
eigenvalue  of  Q(t),  Vt. 

Proof.  If  Q-r  then  II  t K,  so  K is  not  empty.  We  have  shown  that  if  Q 
is  finite  dimensional  then  K is  compact  and  convex  and  invariant  under 
multiplication  by  Q(t)/r(t).  Thus  by  the  Brower  Fixed  Point  Theorem  [9], 

Vt,  3yt  such  that  y£  e K and  Yt(Q(t)/r(t))  = y£.  This  says  YtQ(t)  = r(t)Yt 
so  r(t)  is  an  eigenvalue  of  Q(t).  q 

Corollary  2.9.1.  Q-r  det(Q(t)-r(t)I)=0,  Vt. 

Proof.  This  is  just  a restatement  of  the  theorem.  □ 

Although  it  is  unrealistic  to  check  to  see  whether  r(t)  is  an  eigenvalue 
of  each  Q(t),  theorem  2.9  says  that  one  can  show  that  QTr  by  merely  finding  a 
value  of  t where  r(t)  is  not  an  eigenvalue.  The  following  theorem  is  useful 
in  the  same  way. 

Theorem  2.10.  If  Q is  an  n state  MRP  (n  < «)  and  3t^,t9,  " " tR  such 
that  {QCt^U,  Q ( 1 9 ) U , * * ' , Q(tn)U}  is  a linearly  independent  set  then 

Q-r  <=>nQ(t)  = r(t)n,  Vt. 

Proof.  By  lemma  2.7,  £ is  a one  dimensional  subspace,  so  either  \J  = 9 
or  \J  - {0}.  By  theorem  2.8,  Q-r  3 e \J  so  K must  consist  of  the  single 

vector  {H}.  Since  K is  invariant  under  multiplication  by  Q(CV  r(t)  we  have 
IlQ(t)  = r(t)H.  The  converse  is  a restatement  of  theorem  2.4.  □ 


w 
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Using  the  results  so  far,  a rough  algorithm  for  determining  whether  a 

>KP  is  equivalent  (or  weakly  lumpable  ) to  a renewal  process  can  be  formulated. 

First  of  all,  if  a MRP,  Q,  is  equivalent  to  a renewal  process,  the  renewal 

process  it  is  equivalent  to  must  be  r where  r(t)  = HQ(t)U,  and  ” is  the  steady 

state  vector  of  the  embedded  Markov  chain  Q(°°).  The  first  step  should  be  to 

try  to  show  that  Q*r  since  in  general  that  will  be  the  case.  If  Q is  an  n 

state  MRP  choose  t^,t2»  ' ' ' > cn  randomly  or  otherwise  and  compute 

{Q(t.)U,  •••,Q(t  )U).  If  (Q(t,)U,  •••,Q(t  )U)  is  a linearlv  independent  set 
in  in 

(as  it  will  be  in  general) , then  Q-r  if  and  only  if  U is  a left  eigenvector 
of  each  Q(t).  Choose  some  t and  perform  the  multiplication  !TQ(t) . If  this 
product  is  not  r(t)II  then  Q/r.  If  HQ(t)  = r(t)I!  the  chances  are  it  was  not  a 
coincidence.  See  if  II  is  indeed  a left  eigenvector  of  each  0(t)  by  writing 
out  explicitly  HQ(t)  as  a function  of  t.  If  li  is  a left  eigenvector  of  each 
Q(t)  then  Q-r. 


If  (Q(t.)U,  ••*,Q(t  )U}  was  not:  a linearlv  independent  set,  it  becomes  more 
1 n 

complicated.  If  dim{Q(t^)U,  )U}  = 1 check  to  see  if  U is  a right 

eigenvector  of  each  Q(t)  by  summing  the  elements  of  the  rows  of  0(t) . If  the 
row  sums  of  Q(t)  are  the  same  for  all  t then  Q is  strongly  lumpable  to  r which 
implies  equivalence. 

Another  way  of  trying  to  quickly  show  that  Q/r  is  as  follows.  Choose  t, 

HQ(ti) 

and  t^  and  perform  the  multiplication  (II-  f ^ ^ ) Q(t?)U.  If  II  e K then, 
so  is  nO(t1)/r(t1) . But  for  any  v^  and  v^  in  K,  (v^-v?)Q(t)U  = 0 Vt. 

Thus  if  the  result  of  the  multiplication  is  not  zero  then  Q*r. 

If  no  conclusion  has  been  reached  yet  then  either  try  different  values 
of  time  or  attempt  to  find  the  subspaces  S and  V.  Finding  S is  easy,  finding 
\J  is  much  more  difficult.  The  next  section  gives  a more  abstract  description  of 


1/  and  K. 
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3.  Equivalence  as  a Homomorphism.  Let  Q be  an  n state  MRP  (1  < n <_  =°)  with 
steady  state  vector  R • Associated  with  each  t e [0,®]  is  a matrix  0(t). 

Let  d be  the  ring  of  matrices  generated  by  {Q(t)},  t e [0,=°].  For  each 
probability  vector  y in  Rn  we  have  a map  : 0 — R where 

(4)  F (A)  =YAU. 

A necessary  and  sufficient  condition  for  equivalence  can  now  be  written  in  a 
very  simple  form. 

Theorem  2.11.  Q~r  ~^>  F,.  is  a homomorphism. 

Proof.  ( =^>-)  If  Q-r  then  Vn,  t ,t,,  ••  •,  t RQ(t  ) Q(t,)  • • • Q(t  )U  = 

Li.  n Li.  n 

(JlQCt^U)  (JIQ(t2)U)  •••  (HQ(tn)U).  Also  if  A^A,  £ 0 then  n(A:+A,)U  = .'A  U + :.A,U 
so  F_  is  a homomorphism. 

( ■ — ) If  F„  is  a homomorphism  then  Vn,  t^,t?, 

HQ(t.)  0(t,) • ■ • Q(t  )U  = (HQ(t.)U)(nQ(t,)U)---(nQ(t  )U)  = r(t.)r(t,) • • -r(t  ), 
Li  n l i n Li.  n 

so  Q-r.  □ 

Theorem  2.11  says  that  the  question  of  whether  or  not  a MRP  is  equivalent 
to  a renewal  process  is  identical  to  the  question  of  whether  a certain  map 
from  a matrix  ring  to  the  real  line  is  a homomorphism.  If  there  were  a good  way 
of  deciding  when  there  are  homomorphisms  between  matrix  rings  and  the  reals 
the  analysis  of  equivalence  between  simple  MRP's  and  renewal  processes  would  be 
complete.  This  is  certainly  a deep  question  that  will  demand  more  research. 

The  maps  {F^}  also  give  an  alternate  characterization  of  the  sets  K and  l’. 


Theorem  2.12. 


y £ < 


F is  a homomorphism. 


Proof . ( =*>)  If  y e K then  U e K , so  Q-r.  Thus  F„  is  a homomorphism. 
But  K C.  S , and  from  the  definition  of  S,  F„  = F . 

(^=)  Let  T = {v:vO(t)U  = (vU)YQ(t)U),  and  let 

£'.'  = {w:  w=  yQ(t,)  Q(t,)  •••  Q(t  ) for  some  n,t.  ,t9,  • • *,  t }. 

l i n I z n 

U'  is  a subset  of  T;  for  if  not,  there  would  be  some  t^,t2,  such 

that  yQ(t^)  Q(t0)**"  Q(cn)  ^ T.  But  this  would  imply  that  for  some  t, 

yO(t1)Q(t2)  • • • Q(tn)  Q(t)U  ^ (yQ(t^)  • • • Q(tn>U)  (yQ(t)U)  which  says  that  F^  is 

not  a homomorphism.  The  steady  state  vector  must  be  in  the  closure  of  L* 

since  II  = lim  Y0n(°°),  thus  Ve  > 0,  3w  e U*  that  satisfies  |w-E |U  < e. 
n -*■ 00 

Also  w can  be  chosen  so  that  wU  = 1 since  yQn(®)U  = l,  Vn.  Thus 

j.1Q(t)U  - yQ(  t)  U j = |EQ(t)U  - wQ(t)u|  = |n  - w|Q(t)U  <_  |n- w|u  < e. 

Since  c was  arbitrary,  HQ(t)U  = yQ(t)U  so  H e T . But  if  II  e T then  T = S, 

SO  If  E K.  Q 

If  we  allow  F^  to  be  defined  for  any  vector  y in  Rn,  then  by  a similar 
argument  it  can  be  shown  that  1/  is  the  set  of  all  vectors,  v,  that  make  F^  a 
homomorphism. 


4.  Examples . 


EXAMPLE  1.  Disney,  Farrell  and  DeMorais  [ 3 ] show  that  the  output  of 
an  M/D/1/1  queue  is  a renewal  process.  The  results  obtained  thus  far  allow 
for  a quick  verification  of  this  fact. 

The  output  process  from  an  M/D/1/1  queue  is  a two  state  MRP  with  kernel 


!00^t') 

Qoi<t) 

10(t) 

Qll(t) 

Q(t)  = 


' 

. 


1 
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where  Q _ ( c)  is  the  probability  that 

(A)  a customer  departs  at  time  zero  leaving  i customers  in  the  queue,  and 

(B)  the  next  departure  is  before  time  t,  and  when  that  customer  leaves 
there  are  j customers  left  in  the  queue. 

If  arrivals  are  Poisson  with  rate  A,  and  the  service  times  are  deterministic 
with  rate  d,  then 


0,  if  t < d , 

w*>  = j Y 


. , -As  -Ad  , _ -Ad  -At  . , _ , 

jAe  e ds  = e -e  , if  t > d , 


V°  ■ 


0 , if  t < d , 
t-d 

Ae“Xs  (l-e"Xd)ds  = (1  - e~Xd) (1  - e~X(t~d) ),  if  t > d 


= 


0 , if  t < d , 


e Xd  if  t > d . 


Qn(t)  = 


0 , if  t < d , 


1 - e Xd  , if  t _>  d. 


Thus, 
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e~Ad  (l-e"A(t“d)) 


(l-e~Ad)(l-e-A(t-d)) 


Q(t)  = 


-Ad 


1 - e 


-Xd 


1 (t) 
(d ,“) 


where 


1 (t)  = 

(d,oo) 


0,  if  t < d, 


1,  if  t > d 


The  embedded  Markov  chain  is 


Q(oo) 


-Xd 


-Xd 


1 - e 


-Xd 


1 - e 


-Xd 


so  the  steady  state  vector,  II  = (e  Ad,  1-e  Ad) . Performing  the  multiplication 
IlQ(t),  we  get 


IlQ(t)  = ((l-e“At)  e"Ad  , (l-e'At)(l-e~Ad))  1 (t)  = (l-e"At)  1 (t)n. 


(d ,°°) 

By  theorem  2.4  we  know  that  Q~r  where  r(t)  = (1-e  At)  1 (t) 

(d,°°) 


(d ,°°) 


EXAMPLE  2.  Burke's  Theorem  [ ]_  ] implies  that  the  output  from  a steady  state 
M/M/1  queue  is  a Poisson  process.  The  output  process  is  a MRP  with  kernel  Q(t) 
where  Q^Ct)  is  defined  exactly  as  in  the  first  example,  except  that  in  this 
case  i and  j range  over  all  the  nonnegative  integers.  If  the  service  rate  is 
u,  and  the  arrival  rate  is  1 then  Q(t)  has  the  form 
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q0(t) 

qx(t) 

q2(t) 

q3(t) 

q4(t> 

f0(t) 

fx(t) 

f2(t) 

f3(t) 

f4(t) 

0 

f0(t) 

fx(t) 

f2(t) 

f3(t) 

0 

0 

f0(t) 

f1(t) 

f2(t) 

0 

0 

0 

f0(t) 

fx(t) 

0 

0 

0 

0 

f0(O 

where 


V') 


qj(t)  = 


(As) J -As  -us 


i ! 


e ye  ds 


AjU 


(A  + y ) 


j+1 


- yAj  e-(X+y)t 


J-k 


k=o  ( j-k) ! (A+y) 


k+1  ’ 


Ae  f j (t-s)  ds 


(A+y) 


(l-."1')  - X J+1  e'1'  1 


j+1 


i n /-x  , \k+l  i-k 
k=0  (A+y)  yJ 


1+1  j j-k  -(A+y)  t j-k-p 

+ xJ+1  l l 


k-0  p=0  (j  -k- p) ! (X+p)k+^  pp 


Because  of  the  special  structure  of  Q(°°) , it  can  be  shown  that  the  steady  state 
-1  2 3 

vector,  n = (1—  p)  (l,p,p  , p , •••)  where  p = A/y.  Performing  the  multiplication 
ITQ ( t ) we  get 


i 
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(HQ(O).  = (1-p)"1  (q . ( t ) + l f ,(t)  Pk+1) 

J J k=0  J k 

= (l-e"Xt)(l-p-1)p j = (l-e"Xt)  n.  . 

This  says  that  the  steady  state  output  is  a renewal  process  with  distribution 
(1-e  ). 


EXAMPLE  3.  Now  consider  the  M/M/l/N  queue.  The  output  from  this  queue 
is  an  N+l  state  MRP  with  kernel  Q(t)  = 


0 *0<t> 


where  q^(t)  and  f^(t)  are  as  in  t*ie  last  example.  The  steady  state  vector 

r,N  1-P  .2  .3  N-l  , . . . N . e“ptpN+1 

II  = nrr  (1,P  P P , •**,P  , k),  where  k = p + 


1-P 


N+l 


1 - P 


If  the  steady  state  output  from  the  M/M/l/N  queue  is  a renewal  process 
then  the  renewal  process  would  have  to  have  distribution 


rN(0  . n%N(t)U  - 1- 

1-p 


N+l  - Pt 

P e 


1 - 


N+l 
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v e t’ 


Assume  ” si/.  Then  Yx,  J!‘  Qv(x)  £ V . Since  1/  satisfies 

N 

-N.  . . 

v N N 

^>  ( - n ) _L  QN(t)U,  Yt,  we  must  have  ( — r ^ - TW  ) Q^CtK’  * 0 , V t. 

N 


Q^(t)U  has  the  form  (a,b,b,b,  • • • ,b)  where  a and  b are  positive,  thus 


< "-r^O  - n > VC>» 


/ i-e 


-Xx 


1_0N+1  rN(x) 


, . -Xx  H-l  . 

, „ . l-o  , 1-e  in,  v i 

Da+ ~ ( — _1)b  i c 


1-CN+1  ' rH(x) 


i-1 


1-P  . l-e“Xx  f H . -Ux  PN+1  \ ,N\ 

( — - 7TTT  (P  + e -r—  ) - C )b  . 


1-P 


N+l 


rN(x) 


1-P 


_ Xx 

Since  rN(x)  < 1-e  , each  term  in  the  expression  for 

nNQ.7(x)  m v 

( r~7 nr  ) Q (t)U  is  strictly  positive,  so  It  t V.  Thus  by  theorem  2.8, 

rjjW  N 


the  output  of  an  M/M/l/N  queue  is  not  equivalent  to  any  renewal  process.  (See  [ 3 


5.  Equivalence  Between  Finite  State  MRP's.  Let  Q be  a k state  MRP  with 
steady  state  vector  U , let  Y be  an  m state  MRP  (m  < k) , and  let  {A^.A^  •••  A 
be  a partition  of  the  states  of  Q.  Let  II  be  an  m x k matrix  defined  by 


0 , if  j i A±  , 


if  j t Aif 


and  let  U be  a k * m matrix  defined  by 

0 . if  i 4 Aj  , 

1 , if  i e Aj  . 


satisfy  (2)  but  all  attempts  to  find  a counter  example  have  failed  so  far. 
We  can  show  that  weak  lumpability  implies  equivalence,  though. 


A 


i 


ki 
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Since  the  definition  of  equivalence  between  finite  state  MRP's  is 
analogous  to  the  definition  of  equivalence  between  a MRP  and  a renewal  process, 
one  might  assume  that  the  conditions  for  equivalence  would  be  similar.  First 
of  all,  by  theorem  2.13  any  sufficient  condition  for  weak  lumpability  will 
also  be  a sufficient  condition  for  equivalence,  thus  we  have 

Theorem  2.14.  If  Vt,  TQ(t)  = Y(t)H  , then  Q - Y. 

Theorem  2. 15.  If  Vt,  Q(t)y  - UY(t)  , then  Q ? Y. 

Theorems  2 . 14  and  2 . 15  can  be  proved  the  same  way  theorems  2.4  and  2.5  were 
proved.  They  can  also  be  found  in  Serfozo  [11]. 

Assume  that  for  each  t,  Y(t)  is  an  invertable  m * n matrix  (m  < °°) , and 
Q(t)  is  a k x k matrix  (m  < k < =°) . Q(t)  need  not  be  invertable.  Let  M be 

the  set  of  all  m x k matrices,  M,  with  >_  0 and  MU  * al  where  a is  a scalar. 

In  other  words  M must  have  the  form 

M - 

and  each  row  sum  must  be  the  same. 

If  Y(t)  * JlQ(t)  4 M then  it  must  be  true  that  Y(t)  ^ITQ(t)U  i I which 
says  that  HQ(t)U  i Y(t).  Thus  a simple  necessary  condition  for  equivalence 
is  that  Y(t)-1nQ(t)  £ M. 

Let  S ■ (M  £ H:  MQ(t)U  ■ (MU)  Y(t),  Vt}  and  let  V be  the  largest  subset 
of  S that  is  invarient  under  multiplication  by  Y(t)  ^ on  the  left  and  0(t) 
on  the  right  (i.e.  M z V — ~?>  Vt , Y(t)  HlQ(t)  c ^).  Let  K * (M  e MU  » 1}  . 
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These  definitions  are  analagous  to  the  definitions  in  Section  2 and  not  too 
surprisingly  we  have 

Theorem  2. 16 . Q - Y r e K . 

Proof  • ( ■ ■ »* ) Assume  H i K.  Then  3t1,t-,'*',t  such  that 

^ nQ(t^)Q(t2)  •••  Q(cn)  i But  that  says  3t  so 

that  Y(t  )'1---Y(t1)"1TQ(t.)---Q(t  ) Q(t)U  # Y(t  )-1  •••  Y(t_  )-1  3Q(t. ) • • • 0(t  ) 

But  if  Q - Y then  Y(tn)_1  •••  Y(t1)'1nQ(t1> • • -Q(tn)U  = I so 

3Q(t, ) * ’ *Q(t  ) Q(t)U  # Y(t  ) •••Y(t  )Y(t)  which  is  a contradiction. 
x n l n 

(-^=)  If  H e < 'hen  Yn,  t.,t,,...,t  Y(t  ;-1*  • -Y(t.  )"1nQ(t1 ) — Q(t  )C  - I,  so 

iwinn  i-i  n - 

nQ(t.)---Q(t  )U  = Y(t.) • • *Y(t  ).  o 
i n ~ i n 

In  general  Y(t)  will  not  be  inver table  for  all  t.  This  poses  a serious 

problem.  In  the  renewal  case  this  problem  did  not  exist  since  r(t)  was  a 

scalar,  not  a matrix.  For  equivalence  we  need  HQ ( t ) = Y(t)Ii'  where  in  some 

sense  7.'  acts  just  like  II.  Unfortunately,  if  Y(t)  is  singular  we  cannot  solve 

for  7’  uniquely,  and  we  therefore  have  trouble  defining  the  sets  \J  and  K. 

Another  result  that  carries  over  from  the  renewal  case  rather  easily  is  that 

equivalence  is  identical  to  a certain  ring  homomorphism. 

Let  £ be  the  ring  generated  by  (Q(t)},  t e [0,“],and  let  V be  the  ring 

generated  by  { Y ( t ) >,  t e [0,*].  Define  to  be  _(A)  « TAU. 

jI  ,F  a,  F ~ - 

F _ 

Theorem  2.17.  Q - Y <1  V i-  _ is  a homomorphism. 

*i , r 

The  proof  here  is  identical  to  the  proof  in  the  renewal  case. 

One  result  that  has  no  counterpart  in  the  renewal  case  is 
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Theorem  2.18.  Let  Q,  Y and  D be  k,  m and  i state  simple  MRP's  (k  > m > l)  . 
If  Q - Y and  Y ~ D then  Q G-F  D. 

To  prove  this  theorem  we  need  the  following  lemma. 

F 

Lemma  2.19.  If  Q - Y and  IT  is  the  steady  state  vector  for  Q then  the 

steadv  state  vector  for  Y is  y where  y.  = I . H (i.e.  v = HU). 

1 peF-^i)  P 

Proof.  Since  the  steady  state  vector  is  unique  in  the  class  of  simple 
MRP's,  it  suffices  to  show  that  y satisfies Y Y(“)  = Y. 

Since 

0,  if  j i F_1(i), 

- - ni  1 

'ij  "7"  n » lf  j c F (1)> 

ocF_1(i)  P 

we  have 

yY(«)  = yIIQ(®)y  = nQ(“>)U  * HU  * y.  O 

Proof  of  Theorem.  Let  2 be  the  steady  state  vector  for  Q,  y the  steady 

state  vector  for  Y and  let  ?.p  and  be  the  matrices  induced  by  the  partitions 

F and  G.  We  first  show  that  y II _ » H where  IT  is  the  txk  matrix  induced 

■‘G-F  ~G°F  ~G«F 

on  n by  the  partition  G<»F.  First  of  all,  if  G«F(j)  + i then  (y_H = 0. 

'•G^r  i j 

If  G°F(j)  * i then  since  each  column  of  y^  and  n has  only  one  nonzero  element, 
there  is  some  b such  that  (y^IT^)^  * ^G^ib^F^bj"  From  lemma, 


q£[GoF]-l(i)  4 j ! aeF_1(b)  j qelGoFr1^) 


Thus, 


(v  q ) 

-G-F'ij 


0,  if  GoF(j)  * i, 

, if  G°F( j ) = i , 


qs[GoF]  1(i) 


so 


Iq?F  = -G°F  35  desired- 


If  we  let  U_  be  the  k x m summing  matrix  associated  with  F,  and  U„  be 
**’  r - G 

the  m x l summing  matrix  associated  with  G,  it  is  easy  to  show  that 
UpUg  = yGop  where  UGoF  is  the  k x l summing  matrix  associated  with  G°F. 

Since  Q - Y,  Vn,  tj.,  t2---tn,  • • -Q(tn)UF  = Y^)  • • -Y(tn) . 

Also,  since  Y - D,  Y_Y(t.) • * -Y(t  )U_  » D(T. ) D(t0) • • • D(t  ) . Thus 
■'*  b l n b i j.  n 

^gHpQCtj)  • • •Q(tn)L’FyG  " DCtj^)  • • *D(tn) . But  this  says 
®6.FQ<tl),,,Q<tn)DG.F  * D<t1)*“D(tn)*so  Q ^ D‘  D 


There  are  several  open  questions  on  the  topic  of  equivalence  between 


finite  state  MRP's.  Among  them  are  finding  an  example  of  two  finite  state 
MRP's,  Q and  Y,  that  are  equivalent  but  not  weakly  lumpable;  and  finding 
conditions  for  equivalence  when  the  matrices  Y(t)  are  allowed  to  be  singular 
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1 


I 


If  Q - Y and  Y ? D,  Chen  we  have  shown  that  QG°FD.  Now  suppose  Q - Y and 

Q 

Q - D.  Under  what  conditions  is  there  a partition,  H,  such  that  Y ? D? 
m other  words,  when  is  there  an  H that  makes  the  following  diagram  commute? 


6.  Equivalence  on  a Subset  of  [0,°°].  We  return  now  to  equivalence  between  a 
MRP  and  a renewal  process.  Say  Q is  an  n state  MRP  with  steady  state  vector  I, 
r is  a renewal  process  and  Q - r.  Let  O'  be  another  n state  MRP  that  has  the 
property  that  t £ T implies  that  Q(t)  = Q'(t).  If  t^  T i*l,2,”*k  then 
HQ(t^)  • • •Q(tjc)U  = r(t^)  • • *r(t,  ) . But  since  Q(t)  * Q*(t)  for  t < I we  also 
have  HQ' (t^) • • "Q' (t^)U  = r(t^) • • -rCt^) . Although  it  is  possible  that  O'  - r 
(in  fact  I!  might  not  even  be  the  steady  state  vector  for  O')  we  do  have  a sort 
of  equivalence  between  Q'  and  r on  [0,T]. 

The  motivation  behind  this  section  is  that  in  general  a MRP,  Q,  is  not 
equivalent  to  any  renewal  process  and  therefore  computing  probabilities  of 
successive  interdeparture  times  involves  matrix  multiplication.  If  there  is 
some  BCR  such  that  if  t^,t0,--  s B then  PCT^  _<  <_  t^)  = 

P(T^  < t^)  P(T^  < t^)  • • • P(T^  ^ t^) , computing  probabilities  would  only  involve 
scalar  multiplication  (so  long  as  all  the  times  are  chosen  from  B) . Also,  it 
seems  interesting  to  consider  the  concept  of  a MRP  acting  like  different 
renewal  processes  on  different  subsets  of  [0,°°]  . 


La 
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Definition  2.L,  Let  Q be  an  n state  MRP  and  let  r be  a renewal  process. 
We  say  Q - r on  A if  3y  such  that  y >_  0 and  Yk,  and  Vt^,*--,  t^  c A, 
YQ(t1)-*-Q(tk)U  = r(tl)--T(tk). 

Notice  that  if  Q - r on  A then  Q - r'  on  A for  any  r'  that  satisfies 
r'(t)  = r(t),  Vt  t A.  For  an  arbitrary  MRP,  Q,  one  can  always  construct  a 
set  B Ci  R such  that  Q , r on  B for  some  r (although  sometimes  B will  consist 
of  only  one  point).  Choose  some  t c [0,®J.  By  the  Perron-Frobenius  theorem, 
there  is  a largest  positive  eigenvalue,  a(t)  _<  1,  of  0(t)  and  an  associated 
eigenvector  v(t)  that  satisfies  v(t)U  = 1.  Let  B = { s : v(t)  0(s)  = a(s)  v(t)}  . 
Clearly  Q - a on  B. 


Definition  2.5.  If  {A„},  £ t J,  is  a cover  of  [0,®]  (i.e.  U A„  = [0,  = ]), 

s e b 


and  for  each  6 e J there  is  a renewal  process  rc  such  that  Q ~ r„  on  A 
then  call  {A.}  a renewal-cover  of  Q. 

C 


6 


Such  a cover  exists  for  any  Q since  if  necessary  there  can  be  a different 
renewal  process  for  each  t e [0,®].  The  interesting  question  is  whether  or 
not  one  can  find  a finite  or  countable  renewal -cover . If  Q - r then  there  is 
a renewal-cover  with  one  element  (i.e.  Q - r on  [0,®]). 


Theorem  2.20.  Say  Q is  a finite  state  MRP  such  that  Q„(t)  is  a continuous 
function  of  t for  each  i and  j.  Let  a(t)  be  the  largest  positive  eigenvalue  of 
Q(t)  and  let  v(t)  be  its  associated  nonnegative,  normalized  eigenvector.  If 
the  set  (v(t)},  t z [0,®],  is  finite  or  countable  then  so  is  the  smallest 
renewal-cover,  and  there  exists  an  open  interval  Ad  [0,®]  such  that  0 ~ a on  A. 


A 


n 


Proof . Let  ^v^>v2»'’ 

{s:  v Q(s)  = a(s)v  }. 
n n 


} be  a list  of  the  elements  of  (v(t)}  and  let 

(A  } . 9 

n n=l,2,  • • 


is  the  desired  renewal-cover.  Say 
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t^,  t,,,  •••  e A and  - t.  Since  Q(t)  is  continuous  lim  vnQ(t.)  = v 0(t) , 

i -T  OD 

and  lia  v Q(t.)  = lie  a(t.)v  = a(t)v  . Thus  v 0(t)  = a(t)v  so  A is 
i-*oc  “ i i^coin  n n n n 

closed.  By  the  Baire  Category  Theorem  one  of  the  must  contain  an  open 

interval.  q 
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